METHOD FOR IMAGE TEXTURE ANALYSIS 



FIELD OF THE INVENTION 

The invention relates to a computer-implemented method of analysing 
image texture. In particular, it provides a method of applying selected mappings 
to an image to generate a surface that may be analysed to characterise the 
texture of the image. 

BACKGROUND TO THE INVENTION 

Image texture is a useful characteristic for object differentiation. However, 
image texture is a difficult characteristic to describe without using some kind of 
comparison to already familiar objects. Thus, human analysis of image texture 
is highly subjective. A computer based texture analysis method has the potential 
to provide a quantitative, and therefore more useful, measure of texture. 

Once a textured object is in digital form the problem of analysis becomes 
a mathematical problem rather than the verbal problem of description. The 
mathematical problem has proven to be no less intractable. 

The problem of image analysis has traditionally been approached as an 
analysis of visual images or pictures of macroscopic objects, such as crops or 
forests, or microscopic objects such as stained tissues or cells. However, the 
principles of texture analysis are applicable to any image be it visual, acoustic 
or otherwise. Image texture can be qualitatively evaluated as having one or 
more of the properties of fineness, coarseness, smoothness, granulation, 
randomness, lineation or being mottled, irregular or hummocky. These are 
relative terms, a useful analysis of the characteristics of an image must include 
an absolute measurement of image properties. 

In general terms, the approaches to texture analysis can be divided into 
three primary techniques, these are : statistical, structural and spectral. 



Statistical techniques include first order approaches such as autocorrelation 
functions and higher order approaches such as gray-tone spatial-dependence 
matrices (see Haralick, Shanmugan & Dinstein; "Textural features for image 
classification", IEEE Transactions on Systems, Man and Cybernetics SMC-3(6), 
610-621; 1973). Structural techniques include the use of primitives and spatial 
relationships (see Haralick; "Statistical and structural approaches to texture", 
Proceedings of the IEEE 67(5), 786-804; 1979). The main spectral technique 
is Fourier analysis. 

There are five steps in image processing. These steps are image 
acquisition, pre-processing, segmentation, representation and description, and 
recognition and interpretation. The invention falls into the step of representation 
and description. Persons skilled in the art will be aware of existing techniques 
for performing the remaining four steps of image processing. 

A general explanation of the prior art relevant to texture analysis can be 
found in Chapter 9 of "Vision in Man and Machine" by Levine, M; McGraw-Hill, 
Kew York (1985). Despite research in the area of texture analysis no single 
method has been developed that completely describes texture either verbally 
or mathematically. 

Abdel-Mottaleb makes mention of texture analysis in United States Patent 
No 5768333, as a technique for second stage discrimination of objects. While, 
Abdel-Mottaleb makes use of a large number of gray level thresholds, image 
texture is then characterised by computing many features of each connected 
component in the resulting threshold separately, no 2-dimensional surface is 
formed. 

OBJECT OF THE INVENTION 

It is an object of the present invention to provide a method of image 
texture analysis to usefully describe observed image texture. 
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Further objects will be evident from the following description. 

DISCLOSURE OF THE INVENTION 

In one form, although it need not be the only or indeed the broadest form, 
5 the invention resides in a method of analysing the texture of an image, including 

the steps of: 

mapping the image into a first set of binary representations of the image, 
wherein each representation in the set corresponds to a transformation by a first 
monotonically varying operator; 
10 mapping each binary representation of the first set of binary 

representations into a further set of binary image representations using a 
second monotonically varying operator, thereby forming a matrix of binary image 
representations; 

transforming the matrix by allocating a scalar value to each binary image 
15 representation; 

arranging the scalar values in a two dimensional space against 
parameters of the first monotonically varying operator and the second 
monotonically varying operator to form a first array of scalar values; and 

identifying defined texture characteristics from the resulting first array of 

20 scalar values. 

By monotonic operator is meant an operator acting on an image, the 
result of which only decreases (never increases) , with an increase in its 
parameter, or with a decrease in its argument. 

In preference, the first monotonically varying operator is a threshold 
25 operator wherein the threshold operator applies a threshold value that varies 

monotonically from a minimum value to a maximum value to produce the first set 
of binary representations. 

In preference, the second monotonically varying operator is a spatial 
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operator wherein the spatial operator applies an opening that varies 
monotonically from a minimum value to a maximum value to produce the further 
set of binary representations. Suitably, the second monotonically varying 
operator is a morphological opening that removes all image objects below a 

5 defined size. 

The method may include the further step of calculating a second array of 
scalar values derived from the first array of scalar values by taking at each point 
the minimum of the first difference in a first direction and the first difference in 
a second direction. 

10 The step of identifying defined texture characteristics may include the 

further step of comparing the first or second array of scalar values to a 
knowledge base of first or second array values known to indicate defined texture 
characteristics. 

The method may include the further step of calculating one or several 
15 texture feature values by computing linear sums of the values in the first or 
second arrays of scalar values. In preference, the coefficients of these linear 
sums are determined by a principal components analysis on the data. 

In a further form, the invention resides in an image analysis apparatus 
comprising: 

20 storage means for storing a digital representation of an image; 

processing means for : 

mapping the image into a first set of binary representations of the 
image, wherein each representation in the set corresponds to a 
transformation by a first monotonically varying operator; 

25 mapping each binary representation of the first set of binary 

representations into a further set of binary image representations 
using a second monotonically varying operator, thereby forming 
a matrix of binary image representations; 
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transforming the matrix by allocating a scalar value to each binary 
image representation; 

plotting the scalar values in a two dimensional space against 
parameters of the first monotonically varying operator and the 
5 second monotonically varying operator to form a first array of 

scalar values; and 

identifying defined texture characteristics from the resulting first 
array of scalar values; and 
display means for displaying the first array of scalar values and identified 
10 texture characteristics. 



BRIEF DETAILS OF THE DRAWINGS 

To assist in understanding the invention preferred embodiments will now 
be described with reference to the following figures in which : 
15 FIG 1 is a flow chart showing the method of image texture analysis; 

FIG 2 shows the array of values of the granold surface; 
FIG 3 displays a granold surface; 

FIG 4 shows an example of the application of the method to an image; 
FIG 4a shows a 50x50 pixel gray level image; 
20 FIG 4b is a binary image produced by thresholding the image of FIG 4a; 

FIG 4c is the image produced by opening the image of FIG 4b with a disc 

shaped structuring element of radius 1 ; 
FIG 4d is the image produced by opening the image of FIG 4c with a disc 

shaped structuring element of radius 2; 
25 FIG 4e is a binary image produced by thresholding the image of FIG 4a 

using a different threshold from that used to produce the image of 

FIG 4b; 

FIG 4f is the image produced by opening the image of FIG 4e with a disc 
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shaped structuring element of radius 1 ; 
FIG 4g is the image produced by opening the image of FIG 4f with a disc 

shaped structuring element of radius 2; 
FIG 5a shows the granold surface derived from the example of FIG 4; 
5 FIG 5b shows the granold spectrum derived from the granold surface of 

FIG 5a; 

FIG 6a shows a gray level image in a first example of the application of 
the method to the characterisation of ammonium sulphate pellets; 

FIG 6b is a plot of a granold surface produced by thresholding the image 
10 of FIG 6a; 

FIG 6c is a granold spectrum corresponding to the granold surface of FIG 
6b; 

FIG 6d is a threshold/radius location of the maximum value of the granold 
surface of FIG 6b; 

15 FIG 7a shows a gray level image in a second example of the application 

of the method to the characterisation of ammonium sulphate 
pellets; 

FIG 7b is a plot of a granold surface produced by thresholding the image 
of FIG 7a; 

20 FIG 7c is a granold spectrum corresponding to the granold surface of FIG 

7b; 

FIG 7d is a threshold/radius location of the maximum value of the granold 
surface of FIG 7b; 

FIG 8a shows a gray level image in a third example of the application of 
25 the method to the characterisation of ammonium sulphate pellets; 

FIG 8b is a plot of a granold surface produced by thresholding the image 
of FIG 8a; 

FIG 8c is a granold spectrum corresponding to the granold surface of FIG 
8b; 

30 FIG 8d is a threshold/radius location of the maximum value of the granold 
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surface of FIG 8b; and 
FIG 9 is a block diagram of an apparatus for working the invention. 

DETAILED DESCRIPTION OF THE DRAWINGS 
5 In the drawings, like reference numerals refer to like parts. Referring to 

FIG 1, the flow diagram indicates the steps of a method of texture analysis. The 
method is preferably implemented in a computer, persons skilled in 
programming will be able to readily produce appropriate code in a suitable 
language to implement the method. 

10 The flow diagram is directed to the situation in which the monotonic 

mappings are gray level thresholding and spatial opening. The example is two- 
dimensional but the method is applicable to multi-dimensional applications. A 
mathematical description is presented later which generalises the specific 
example to a wider range of situations. 

15 In the flow diagram, the gray level image to be analysed is designated as 

X. A threshold operator <3> v is applied at a first value of I = i min . Typically, an 8-bit 
gray level image might first be thresholded at gray level 0. A binary image <D ( (X) 
is obtained by this thresholding operation where pixels having a gray level 
above the threshold will have a value of one and all other pixels have a value 

20 of zero. Applying spatial operator x ¥ ] then morphologically opens the binary 
image. 

In the example described here, a set of disc shaped structuring elements 
of increasing radius is used as the morphological opening. Thus, the smallest 
value of j = j min corresponds to the first member in the set, which has the smallest 
25 radius. All pixel clusters having a radius less than j are set to zero and the rest 
are retained. The remaining area of the resulting binary image ^O^X) is then 
calculated (for example, by counting all pixels having a value of one) to give a 
value WjO^X). This value is stored in a two-dimensional array G x in a position 
corresponding to the i m threshold operator versus the f 1 spatial operator. 
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The value of j is iterated and the next spatial operator is applied to 
0,(X) and the value 8Yj +1 O i (X} is calculated and stored in the array at location 
G x (i,j+1). The iteration and application of the spatial operator continues until j = 
j max . The value of j max may be arbitrarily chosen or may be the value of j such that 
5 SYjroaxOjfX) = 0. This process of continually opening a binary image with a 

suitable set of structuring elements and measuring the remaining area produces 
a morphological granulometry of the binary image. 

Once the granulometry of the first binary image has been completed the 
original gray level image is thresholded for the next value of I. The 

10 morphological granulometry for the 1+1* binary image is then formed and 
entered into the granold surface array. The process is repeated, as shown in 
FIG 1 , until I = i max . The value for I may be chosen arbitrarily or may be the value 
of the highest gray level threshold possible in the image. For an 8-bit gray level 
image this would be 255. 

15 The resultant array is represented in FIG 2 and is referred to as the 

topographical granold surface. The array has (i max - i mjn ) x (j max - j min ) elements 
that can be plotted as a surface in the manner shown in FIG 3. Corners appear 
on the surface that indicate a relationship between size and gray level of 
granules in the original image. An idealised corner is shown in the surface 

20 G x (i,j) of FIG 3 at a position corresponding to l=i c and j=j c . Further processing, 
for example by taking at each point the minimum of the first difference in the I 
direction and the first difference in the j direction of the granold surface, can be 
performed to form the granold spectrum (also called the granold korner surface). 
As shown in the following figures, further processing can result in a spectral-like 

25 result that simultaneously reveals the distribution of size and gray level of 
granules within the original image. 

FIG 4 shows an example of a number of images produced by the method 
described above, FIG 4a shows a 50x50 pixel gray level image. FIG 4b is a 
binary image produced by thresholding the image of FIG 4a at a threshold level 

30 of 172. The image of FIG 4b is opened with a disc shaped structuring element 



of radius 1 to produce the image of FIG 4c. The image of FIG 4c is opened with 
a disc shaped structuring element of radius 2 to produce the image of FIG 4d. 
This process is continued with increasing radius until the remaining image is 
reduced to zero. The original image is thresholded at a gray level of 209 to 
produce the binary image of FIG 4e. Disc shaped structuring elements of radius 
1 and 2 are used to produce the images of FIG 4f and FIG 4g respectively. 

FIG 4 clearly demonstrates that the number of remaining pixels 
decreases with increasing threshold and increasing radius of the spatial 
operator. The actual values of the granold surface calculated for each image are 
G x (172,1)=1296, G x (172,2)=1210, G x (209,1)=133 and G x (209,2)=0. 

The complete granold surface G x (i,j) for the image of FIG 4a is displayed 
in FIG 5a. The surface was constructed using a threshold operator that applied 
gray level thresholds from 0 to 255 and a spatial operator that applied a set of 
disc shaped structuring elements with radii from 1 to 16. The plot has been 
normalised to the original image area of 2500 pixels. The corresponding granold 
spectrum K^ij) is shown in FIG 5b. The granold spectrum provides a spectral- 
like plot that characterises the original image in a reproducible and objective 
manner. 

EXAMPLE 

Characterisation of matter by pellet size and colour intensity may be 
applicable to a variety of industries, such as confectionery, pharmaceuticals and 
fertilisers, for quality control. FIG's 6-8 provide an example of the application of 
the invention to the characterisation of ammonium sulphate pellets. 

In each of FIG's 6-8, image (a) is a gray level image produced by a video 
camera and frame grabber as ammonium sulphate pellets fall from a drum 
granulator in a fertiliser plant. The pellets of FIG 6 and FIG 7 are the same size 
but the pellets of FIG 8 are larger. Green food dye was added to the pellets to 
provide colour variation with FIG 6 being the darkest and FIG 8 being the 
lightest. 
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Image (b) of each figure is a plot of the granold surface produced by 
applying a threshold operator that applied gray level thresholds from 0 to 255 
and a spatial operator that applied a set of 1 6 disc shaped structuring elements. 
Image (c) is the corresponding granold spectrum. 

5 Further processing of the granold spectrum involved identifying the 

threshold/radius location of the maximum value of the surface. The value 
indicates the gray level and size of the majority of the pellets in the image. It will 
be appreciated that statistical analysis of the granold spectrum can characterise 
the pellets by the mean and standard deviation of the size and colour of the 

10 pellets. Such an analysis provides a useful standard for quality control. 

It will be appreciated that the foregoing examples are limited to two- 
dimensional space but can be generalised to multi-dimensional images to 
produce a similar granold surface and granold spectrum. Furthermore, the 
operators that produce the mappings are not limited to threshold, opening size 

15 and area. Other suitable operators may include contrast dynamic, attribute 
opening, etc. 

The following generalised description is presented mathematically in a 
manner that will be understood by those skilled in the relevant art. 



20 MATHEMATICAL DESCRIPTION 

In order to describe the method it is necessary to define certain notations. 
In the following, "R + " denotes the non-negative real numbers (including zero), 
"Z" the integers {.,.,-1,0,1,...} and "N" the natural numbers {0,1,2,...}. 

We define 

25 {F};Z n Z w (1) 

Here {F} represents the general class of m-integer valued functions on n- 

dimensional space. We would typically use such a function to represent a gray 
scale 2-D image, in which case we would write, for example, "let X eF be a 
2-D gray scale image . . .". This statement implies that n=2 and rn=1 . We keep the 
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dimensions of the image representation general (at n and m) so as not to 
unnecessarily limit the scope of the method to 2-D images. The domain of the 
image X e F will be denoted by Dom(X) . The domain is the set of al points 

in Z" for which X is defined. 
5 Next we define, 

B e P(X") (2) 

Where P is the powerset operator, so P(Z") is the set of all subsets of Z" . B 

represents the general class of binary images on n-dimensional integer space. 
We would typically use such a notation to represent a 2-D binary image where 

10 we would write, "let be a 2-D binary image ..." which statement implies that n+2. 

Again, we keep the dimensions of the image general at n so as not to 
unnecessarily limit the scope of the method to 2-D images. The domain of the 
binary image B e B will be denoted Dom(B) . 

Based on the image spaces defined above we now define the mappings 

15 or operators used to form the granold surface. Firstly, we define, 

<D:F->B (3) 
This function to set mapping takes a gray scale image and returns a binary 
image. As an example let leFbea 2-D gray scale image, so we could 
write, 5=0 {X) . The resulting image B e B is a 2-D binary image. 

20 The thresholding of regions based on region minima or maxima values, 

the thresholding of regions based on contrast dynamic values, and the gray level 
threshold operation, well known in image processing, are all examples of this 
type of mapping. 

Secondly we define, 

25 ¥:B->£ (4) 

This set to set mapping, which we will call an "operator", takes a binary image 
and returns a binary image. As an example, let B e B be a 2-D binary image so 
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we could write, A = Y (B) . The resulting image A e B is also a 2-D binary 
image. 

The binary AND operation, the attribute opening operation, and the 
morphological binary opening operation, also well known in image processing, 
5 are all examples of this type of mapping. 

It is the intention that these mappings preserve the domain or support of 
the images in all cases, that is, 

Dom($(X)) = Dom{X) (5) 

and 

10 DomQ¥ (£)) = Dom(B) (6) 

and with the composition of these mappings' 

Dom(V 0 (X)) = Dom(X) (7) 
Lastly, we define the set measure, 

£B->R + (8) 

15 This set to scalar mapping, which we will call a "measure", takes a binary image 

and returns a non-negative scalar value. The area (number of object pixels) of 
a binary image is an example of this type of mapping, so we could write, for 
example, area = S(B) . 

The next step is to parameterise the mappings 3> and to give families 
20 of mappings and operators indexed on non-negative integer parameters / e N . 
Therefore we have families, 

{0,}:^-* B (9) 

and, 

{¥,}:]*-> B (10) 

25 It is important for the method that the above mappings posses additional 

mathematical properties as follows: 
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Property 1 - Monotone Decreasing for Parameter. 

The families of mappings 0 1 and x ¥ j must be "monotone decreasing" 

with respect to their parameter I. Also, to ensure that (in the limit) the mappings 
extract all the information from the images, there should exist values M and N 
(which may depend on the images) such that for i>M and i>N respectively, the 
appropriate mappings produce the empty-set, 0 . In contrast, we should also 
ensure that the index zero mappings have the least possible effect on the image 
and respectively produce the domain of the image or reproduces the image 
itself. 

In mathematical terms, for the mappings 0 . and for any X eF , 



Property 2 - Monotone Increasing for Argument 

The families of operators (set to set mappings) x f i need to have some 

additional properties to ensure the granold surface is well behaved. Firstly, 
Y f needs to be "anti-extensive" and to remove detail from the binary image 

B e B , that is, for all / e N 



i = 0 ^ ® i (X) = Dom(X) 
0<i<j<M ^<D.(I)c$.(I) 
i> M => O t (X)=0 



(11) 
(12) 

(13) 



Likewise, for the operators \P, and for any A e B , 

i=0 ^ X H.(A)=A 
0<i<j<N ^(^cT,(ii) 
i> N => ¥ I (A)=0 



(14) 
(15) 
(16) 



W=5 



(17) 
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Secondly, the operators x ¥ i must be "monotone increasing" with respect 

to their arguments, the binary images, that is for the binary images A, B e B for 
a fixed integer i e N 

A^B^ ¥,(,4)c (18) 
5 Property 3 - Measurement 

Finally, the measure 8 , to represent the usual sense of measuring 
something, must be : (I) zero if the object to be measured is empty, 
8(0) =0 (19) 

and (ii) increasing, that is, 
10 Ad 8(A) < 8(B) (20) 

Now we consider the composition of these mappings as follows' 
(<S¥/D,.):F-> R + (21) 

This function to scalar mapping, which we have previously termed a "granold" 
(granulometry of threshold), takes a gray scale image and returns a non- 
1 5 negative scalar which depends on the values of the parameters / and j (and of 
course the image itself). 

To show the dependence on / and j explicitly, we choose to denote the 
granold G x (iJ):W -> R + of an image X e F as a 2-D function, 

def 

G x (iJ) = (8^ j ^ i )(X) (22) 

20 If G is represented as a function on two dimensions or a surface then 

regions /and j both have a large effect on the value of 6 will appear as "corners" 
in the surface. The location and "sharpness" of these corners will reveal the 
particular values of / and y'that have a large effect on the value of 8. 

The following first differences are useful for analysing G. These 

25 differences approximate the directional derivatives of the granold and will be 
large wherever there is a rapid fall in the granold in the corresponding 
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directions. 



def 



V x {iJ)=G x {iJ)-G x {i+\,j) 



(23) 



def 



(24) 



The quantity, 



def 



K x (i,j) = min(h x x (ij),^ x (i,j)) 



(25) 



is useful to determine where the granold surface has simultaneous large first 
differences in both the fl> . and the v ff j directions, and produces the granold 

spectrum. 

To classify textures we may compute the texture features, 



where the coefficients, a Wj and p kij and would be found by, for example, principal 
components analysis of the data. 

A suitable apparatus for putting the invention into effect is shown in Fig 
9. An image is acquired with acquisition means 1 . This could be a digital camera 
or digitizer that converts an analogue signal. Pre-processing may occur at 2. 
This may be on-board processing in the acquisition means 1, or may be, for 
example, post-acquisition filtering. 

Signals are passed to a processor 3 that is programmed to work the 
method described above. Required parameters may be stored in ROM 4 and 



(26) 



and/or, 




(27) 
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temporary storage is provided by RAM 5. The result of the image texture 
analysis is displayed on monitor 6. Other data storage means such as disk or 
tape (not shown) may also be connected to the processor 3. The data storage 
may store a knowledge database of previous image texture analyses that can 
5 be used to identify texture characteristics by comparison. 

Persons skilled in the relevant art will appreciate that the method of 
image texture analysis can be applied to a diverse range of situations. The 
inventors have demonstrated the application of the method to fertiliser pellets. 
Preliminary considerations indicate that it is also applicable to analysis of 
10 cervical cells on a Pap smear slide on a microscopic level and skin screening 
at a macroscopic level. Throughout the specification the aim has been to 
describe the preferred embodiments of the invention without limiting the 
invention to any one embodiment or specific collection of features. 

15 



